Due to the specialized ways that database management systems (DBMS) utilize memory to access data, a DBMS typically implements its own memory management techniques rather than relying on more general memory management techniques that are provided by the underlying operating system on which the DBMS runs. For example, a DBMS may expressly request that the operating system allocate to it a portion of memory so that it can manage such memory on its own, thereby avoiding triggering of memory management techniques (disk swaps, LRU page replacement algorithms, etc.) that the underlying operating system may typically use to “over-commit” its available physical memory in an effort to provide running applications a larger “virtual” memory space in which to execute. That is, although the operating system may provide the DBMS a virtual memory space that is larger than the portion of physical memory allocated to the DBMS application, the DBMS application can, using its own memory management techniques, ensure that it utilizes its virtual memory space within the bounds of its allocated physical memory and therefore avoid any operating system level memory management activities that would otherwise adversely affect performance of the DBMS (e.g., untimely writes to swap disk, etc.).
Tuning the DBMS memory has been at the forefront of database research. Recently, autonomic techniques for tuning the memory of DBMS at runtime have been developed for some commercial relational databases. One such technique employs memory pools, each employing specialized paging policies apart from the paging policies of the operating system. Tuning parameters in this technique include the amount of memory to be allocated to these pools and how that memory is to be divided among the various memory pools.
The largest of the memory pools is the buffer pool, which contains the memory pages of database tables that are actively involved in transaction processing. As a transaction modifies rows in database tables, the pages containing these rows are brought into the buffer pool from disk and are modified in place. When the transaction is eventually committed by the DBMS, these “dirty” pages are flushed to disk under the control of the DBMS, for example, by atomically writing a record relating to committed transaction into a write-ahead transaction log on disk to ensure that the transaction's changes are never lost. It should be noted that the DBMS, not the operating system, determines when dirty pages of the buffer pool are written to disk.
In addition, the DBMS, implementing its own memory management, typically maintains its own free list of memory pages and memory page descriptor data structures that are separate and different from any memory management based free memory page lists and data structures maintained by the operating system. Indeed, a memory page that the DBMS may regard as free (e.g., because it has recently completed a database query transaction relating to the data in the memory page) may actually appear to the operating system to be a more important memory page because the DBMS has recently accessed the memory page. As an additional example, memory page descriptor data structures maintained by the DBMS may indicate which memory pages are “clean” and which ones are not. Clean memory pages are those that contain data that matches the corresponding data stored in the database on disk. Because the operating system has no knowledge that the DBMS utilizes portions of its allocated memory as an in-memory cache of the data it stores in the database on disk (e.g., for faster access and query response times), it is not able to similarly characterize the memory pages used by the DBMS.
The DBMS's own memory management techniques referenced above work well when the DBMS is the only application running on a host computer and is able to ensure its own allocation of physical memory. However, when a DBMS is run in a virtual machine that is hosted on a computer with other virtual machines, or the DBMS is run in a physical machine alongside other workloads, the DBMS, as initially tuned, may be forced to run with insufficient physical memory, resulting in performance degradations, e.g., due to disk swaps.